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Introduction. Several controlled vocabularies are used for indexing three journal articles to 
check if better or equal consistency rates are achieved with a list of descriptors than with a 
standard thesaurus and augmented thesaurus. 

Method. A terminology set for library and information Science was used to build a list of 
descriptors with equivalence relations (USE and UF), a standard thesaurus and an augmented 
thesaurus (all the descriptors have scope notes). Subsequently, three articles were indexed by 
selected indexers with varying degrees of experience: on the one hand Library and 
Information Science students and, on the other, professionals from various documentation 
centres. Hooper's measure to find the consistency between pairs of novice indexers and experts 
has been applied. 

Analysis. Data were tabulated and analysed systematically according to pairs of novice 
indexers and experts. 

Results. The tool with the best results is the list of descriptors (39.5% consistency), followed by 
the augmented thesaurus (29.8%) and, with an almost identical value, the standard thesaurus 
(27.5%). 

Conclusion. It is concluded that the list of descriptors in both groups returns better indexing 
consistency but more research is required. 
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Introduction 

Vocabulary control has been revealed as an essential procedure in the organization and retrieval of information. The 
most significant contributions in this field of work are many and varied although the main ones taken here are those 
from Gil-Leiva, 2008 : 118-154. The first contribution was the work done by Charles Ammi Cutter in his famous Rules 
for a printed dictionary catalog published in 1876. It is here that the first rules appear that are in effect today, such 
as the principle of economy, the definition and use of the term headings for both matter and for place and form; re¬ 
sending for synonyms and antonyms; the problem of homonymy; the structure of the subject headings (simple and 
complex); word inversion; syntax (See, See also, etc.) and punctuation marks (commas, brackets, etc.). 

The second contribution was the building of lists of subject headings. Shortly after the contributions of Cutter, the 
American Library Association (ALA) published in 1895 the List of Subject Headings for Use in Dictionary Catalogs 
as an indexing tool for small and medium sized libraries with non specialized stocks. The first Subject Headings Used 
in the Dictionary Catalogs of the Library of Congress appeared in 1909 and took as its main references the 
contributions mentioned above. Although it came into being for internal use for the cataloguers in the Library of 
Congress, it would soon become a reference tool used in indexing in large public and academic libraries and it was 
translated or either totally or partially adapted to other countries and languages, for example, Brazil (1948), Canada 
(1967), Greece (1978), South Africa (1992) or Egypt (1995) among others. 

The third contrihution comes from Mooers, who at the beginning of the 1950s introduced the word descriptor to 
communicate ideas, so distancing himself from particular terminological uses employed in documents and thus 
specifying the subject of the information in an information retrieval context. A follow-on to this was the construction 
of the first lists of descriptors and the first thesauri, like the Dupont Thesaurus (Engineering Information Centre Du 
Dupont 1959), the Thesaurus of Astia Descriptors (United States Department of Defense, i960), or the Chemical 
Engineering Thesaurus (American Institute of Chemical Engineers, 1961), among others. 













The fourth contribution is the provision of national and international norms. Work in this sphere got underway early 
in France, since in 1957 the AFNOR Z 44-070 Catalogue alphabetique de matieres was presented, which was devoted 
to establishing and providing rules for the choice and presentation of subject headings. The first norms for thesauri 
were the French AFNOR Z 47-100-1973 (Norme experimental. Regies d’etablissement des thesaurus monolingues), 
the ISO 2788-1974 (Documentation. Guides for the establishment and development of monolingual thesauri) and the 
ANSI Z39.19-1974 (American National Standard guidelines for thesaurus structure, construction and use). Since 
then, other countries and the ISO itself have been working on and extending the norms until the unification of the 
ISO 2788-1986 and ISO 5964-1985 in the newISO/DIS 25964-1:2010 (Information and documentation—Thesauri 
and interoperability with other vocabularies (Part 1: Thesauri for information retrieval; Part 2: Interoperability with 
other vocabularies). 

The evaluation of controlled vocabularies is an issue of concern for professionals and researchers in the area. The 
evaluation can be performed with the aim of the analysis being the controlled vocabularies themselves so as to study 
their structure, the thematic fields or facets, scope notes, semantic relations, degree of specificity, etc., (intrinsic 
evaluation) or by studying the impact on the information systems which use them both in indexing and retrieval 
(extrinsic evaluation). 

The first evaluation of import was carried out by Cleverdon in the Cranfield Projects ( 10^6 : i960, etc.). Cleverdon 
compared the efficiency of the Universal Decimal Classification, an alphabetical index of subjects, a faceted 
classification scheme and the indexing through uniterms of eighteen thousand documents analysed by three indexers. 
There have been many and varied subsequent studies to evaluate controlled vocabularies, both subject headings and 
thesauri. We have for example the works by Henzler (1978), Fidel (1991 and 1002 ). Betts and Marrable (1991); 

Ribeiro (1996), Gil Urdiciain (1998) and Gross and Taylor (2005.), who studied the advantages and drawbacks of 
indexing and retrieving documents in natural language and in controlled language. 

Another way of evaluating controlled vocabularies, mainly thesauri, is to compare them with each other. Kishida, et 
al. (1988) compared the MeSH (Medical Subject Headings), the ERIC thesaurus, the INSPEC and the Root thesaurus, 
among others, taking as their reference the construction principles, their structure and the information they 
contributed. In contrast, Weinberg and Cunningham (1985) studied the semantic proximity between MeSH and 
Medline, while Pozhariskii (1982) proposed quantifying the capacity or semantic strength of a thesaurus in terms of 
flexibility, economy and universality. Elsewhere, Larsen (1988) analysed the capacities for use of a thesaurus for 
indexing a certain collection of documents. Soler Monreal (2009) evaluated three controlled vocabularies (a list of 
descriptors, a standard thesaurus and an augmented thesaurus in which all the descriptors have scope notes) in 
order to find out if consistency scores higher than a standard thesaurus and augmented thesaurus are obtained with 
a list of descriptors. 

Indexing consistency can be studied as a reference to a single indexer or to several. When a professional indexes the 
same document at different moments in time we speak of intra-consistency or intra-indexer consistency. And when 
several people indexing a document to compare the results or the result of indexing a document by two indexers is 
compared, we speak of inter-consistency or inter-indexer consistency. 

Since the 1960s, numerous and diverse investigations have been carried out on indexing consistency. The main 
conclusion which can be drawn from the tests is that inconsistency is an inherent feature of indexing, rather than a 
sporadic anomaly. Although the tests carried out are very diverse in their methodology, we can say that achieved 
indexing consistency ranges from approximately 10% to 60%. The vast majority of the tests carried out from i960 
until the present time cannot be homogenized because of the methodological diversity used. We only point out here 
some of the variables that hinder their homogenization and only a sample of the tests carried out: 

• Measures: To find the consistency scores between groups of indexers Slamecka & Jacoby (1965) and Iivonen 
(1990) proposed their respective measures. In contrast, for pairs of indexers, other measures were outlined as in 
Hooper (1965.), Lancaster (1968), Rolling (1981) or Saarti f 2002 b although the most commonly used in most 
tests is that of Hooper, c / (a + b - c), where c is the common terms between the two indexers, a is the terms 
proposed by indexer one, and b is the terms proposed by indexer two. 

• Novice versus expert indexers: On some occasions expert indexers were employed as in Lancaster (1968), 
Leonard (1975); on others it was with novice indexers Hudon (1998a and 1998b), Gil-Leiva ( 2002 ): or with 
experts and novices (Bertrand and Cellier 1995 ; Saarti, 2002 : Soler Monreal 2009). 

• Number of indexers: Lancaster (1968) worked with three indexers; Bertrand and Cellier (1995) twenty-five 
indexers; Hudon (1998a and 1998b) twenty-five indexers; Gil-Leiva (2002) twenty-seven indexers; Saarti 
( 2002 ) thirty indexers; and Soler Monreal (2009) sixty-three indexers. 

• Material used: Sometimes work has been with journal articles ( Lancaster 1068 : Leonard 1Q77 : Funk and Reid 
1983; Middleton 108a : Sievert and Andrews iqqi : Tivonen and Kivimaki iqq8 : Leininger 2000 : Gil-Leiva 
2002 ). sometimes with books ( Tonta iqqi : Bertrand and Cellier iqq 1 ; : Gil-Leiva 2001 : Saarti 2002 : Neshat and 
Horri 2006 : Gil-Leiva et al. 2008 : Chen 2008 I and at other times with visual material f Markev 108/1 and Gil- 
Leiva 2002 ): summaries of journal articles have also been used ( Hudon iqq8 : Soler Monreal 2000 ): social 
tagging on delicious.com Kipp 2000 ) and social tagging on CiteULike.org ( Wolfram et al. 2000 ). 

• Number of documents indexed: Lancaster (1968) used sixteen articles; Tarr and Borko (1974.) fifteen items; 


















































Leonard (1975) 100 articles; Funk and Reid (1283.) 760 articles; Markey (1284) 100 documents; Iivonen ( iqqo ) 
ten documents; Siever and Andrews ( iqqi ) seventy-one articles; Iivonen and Kivimaki (1228) forty-nine 
documents; Leninger (2000) sixty documents; Tonta (1221) eighty-two books; Bertrand and Cellier (1995) eight 
books; Gil-Leiva (2001) eleven books; Saarti (2002) five books; Gil-Leiva et al. ( 2008 ! ten books; Chen (2008) 
3,307 monographs; Hudon f iQQ8a and 1228b) twelve abstracts; Monreal Soler (2002) three abstracts. 

• Hawthorne effect: Individuals, when they know that they are being studied behave differently from when they 
do not know they are being observed. In some studies indexers knew their product would be evaluated 
( Lancaster 1068 : Leonard 107 ^; : Bertrand and Cellier looq : Hudon (1228a and 1228b); Gil-Leiva 2002 : Saarti. 
2002 : Soler Monreal 2000 ): and in other studies as the result of indexing the same documents is compared but 
in different information systems could not be this possibility as, for example, two bibliographic databases: 
Middleton ( 1084 ) compares the indexing of ERIS/APAIS and AEI/APAIS; Iivonen and Kivimaki ( 1008 ) 
databases KINF and LISA; or compare the indexing library catalogues like Tonta ( iqqi ) who compared the 
Library of Congress and the British Library; Gil-Leiva ( 2001 ) in thirty-one catalogues of public libraries; Neshat 
and Horri ( 2006 ) National Library of Iran and twelve academic libraries; Gil-Leiva et al. ( 2008 ) thirty 
university library catalogues; Chen ( 2008 ). the National Library of China and the China Academy Library & 
Information System. Finally, duplicate records in information systems. Funk and Reid (1283.) use 760 articles 
indexed in Medline twice; Siever and Andrews ( iqqi ) worked with seventy-one duplicates of the database 
Information Science Abstracts; Leininger ( 2000 ) compared sixty duplicates of the PsycINFO database. 

• Concepts versus terms: In most of the studies mentioned above these are compared with all the indexing terms 
or descriptors derived from a controlled vocabulary, while sometimes comparisons were made with the 
concepts taken directly from the documents to find consistency, such as Iivonen ( iqqo ) or Gil-Leiva ( 2002 ). 
that take both the concepts used as descriptors from Eurovoc Thesaurus. 

Materials and methods 

For this study we built three controlled vocabularies on information science: a list of descriptors with control for 
synonymy; a standard thesaurus and a thesaurus in which all the descriptors have scope notes (augmented 
thesaurus). 

At the time of initiating this research there did not exist in Spanish a thesaurus published on this subject. Hence, we 
began to refine a list of descriptors consisting of 2,756 terms which were in use in the design and maintenance of an 
automatic indexing system ( Gil-Leiva 1QQ7 and 2008 ). Finally, the list was a total of 2,455 terms, of which 1,436 are 
descriptors and 1,019 non-descriptors. A standard thesaurus was constructed from this list. This thesaurus has an 
alphabetic display, another hierarchical one and other types KWOC permuted index. Appendix A shows the first 
terms of the three tools built. 

The thesauri were built with the thesaurus management software MultiTes and following Spanish norm UNE 50-106- 
90 (equivalent to ISO 2788-1986). 
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TC: J02 
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Purchases 

TGI: Acquisition of 
documents 

TG2: Development of 
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TG3: Documental process 

TC: F03 

TGI: Catalogues (information 
sources) 

TG2: Secondary sources 

TG3: Information sources 


Table 1: Descriptors of the standard thesaurus 


Finally, specialized dictionaries are used to add the scope notes to all the descriptors to build augmented thesaurus. 
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Table 2: Descriptors from augmented thesaurus 


After building the three controlled vocabularies, an intrinsic (qualitative and quantitative) evaluation was carried out 
to check that they comply with the recommendations for the compilation of thesauri. The compilation was carried 
out following the parameters proposed by Lancaster ( 2002 ). Gil Urdiciain ( 2004 ) and Gil-Leiva ( 2008 ). It was 
confirmed that the thesauri meet the traditional requisites for compilation of thesauri. 

Later, we decided that the material to be indexed was to be three abstracts of journal articles since these are concise, 
well structured and understandable information sources ( Appendix B ). We then worked on the selection of the 
indexers who were going to use the three indexing languages to index three abstracts of information science articles. 
Finally, we decided that the indexers should have different levels of experience. 

Group 1: Second year information science students 
Group 2: Fourth year information science students 
Group 3: Fifth year information science students 
Group 4: Experienced professionals in document indexing 

The three groups of students already had some theoretical and practical knowledge of indexing and use of controlled 
vocabularies. Each group comprised eighteen people and was divided into three subgroups of six indexers for each of 
the three tools. The exception was Group 4, which was made up of nine professionals for whom indexing is a habitual 
task. The professionals work in documentation centres in public administration (3), communication (3) and 
technological institutes (3). These were also subdivided into nuclei of three indexers per tool. None of the indexers 
were familiar with the indexing languages constructed for the tests, although both the novice and the expert indexers 
had used indexing languages from other fields. Finally, it should be mentioned that it was difficult to find more 
professionals who were available to participate in these types of tests. 

The results of the indexing of the three abstracts were compared pair wise, so novice indexers were compared fifteen 
times for each of the three articles and for each of the three tools being compared - giving a total of 137 comparisons. 
As regards the expert indexers, three comparisons were obtained for each for each of the three articles and the three 
tools under comparison - giving a total of twenty-seven comparisons. 

We used a relaxed, and non exact, system of coincidence to calculate consistency between indexers, as was done in 
Gil-Leiva ( 2001 ) and Gil- Leiva et al. ( 2008 ). A coincidence of 1 (100%), 0.5 (50%) or o (0%) was considered. For 
example, if one indexer consigns librarians and another reference librarians, a consistency of 0.5 is recorded. As a 
general norm, it was considered that a score of 0.5 should be awarded to those non coincident terms that were, 
however, specific of another one, while 1 was given to very similar concepts. 


Indexer 1 

Indexer 2 

Agreement 

Biomedical journals 

Scientific journals 

0.5 

Librarians' 

techniques 

Librarianship 

1 

Databases 

Bibliographical 

databases 

0.5 

Librarians 

Librarians of reference 

0.5 

Scientific journals Scientific publications 
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Table 3: Table of relaxed equivalences between 
descriptors 


Since their beginnings, tests on indexing consistency have used various formulas, among which the most important 
are those used by Hooper f iobO and Rolling f ioSi l. Gil-Leiva f iQQ7 and 20m l. Gil-Leiva et al. (2008) and Soler 
Monreal ( 2000 ). We have used extensively Hooper’s measure of indexing consistency adapted as follows: 

C; =- Tee — 

(A + B) - T co 

where C; is the consistency between two indexings, T co is the number of terms in common between the two indexings, 
A is the number of terms used by Indexer A, B is the number of terms used by Indexer B, and T co is the number of 
terms they use in common. 

Example: 
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Agreement 

1. Librarianship 

1. Librarianship 

= 1 

2. Cite frequency 

2. Medical 


3. Biomedical 

documentation 


journals 

3. Impact factor 

4. Scientific journals 

=0.5 


























Table 4: Equivalences between descriptors 


Results and discussion 

The results from the comparisons carried out to ascertain the consistency for each of the indexing languages 
constructed are organized by groups for the sake of presentation and analysis as can be seen in Appendix C . 

The results of the tests with novice indexers are summarized in the means table below: 



List of 
descriptors 

Augmented 

thesaurus 

Standard 

thesaurus 

Second year 
students 

29.5 

25.9 

29.6 

Fourth year 
students 

39.5 

34.1 

23.7 

Fifth year 
students 

33.3 

35.8 

26.3 

Means 

34.2 % 

31.9 % 

26.5 % 


Table 5: Means of the results of the novice indexers 
with the three tools as % 

However, data for consistency among indexers experts are: 



List of 
descriptors 

Augmented 

thesaurus 

Standard 

thesaurus 

Expert 

indexers 

55.7 % 

23.7 % 

31.3 % 


Table 6: Means of the results for the expert indexers 
with the three tools as % 

We have also obtained the mean for all the consistency obtained, for both expert and novice indexers, as can be seen 
in Table 7. 



List of 
descriptors 

Augmented 

thesaurus 

Standard 

thesaurus 

Second year 
students 

29.5 

25.9 

29.6 

Fourth year 
students 

39.5 

34.1 

23.7 

Fiftht year 
students 

33.3 

35.8 

26.3 

Expert 

indexers 

55.7 

23.7 

31.3 

Means 

39.5% 

29.8% 

27.7% 


Table 7: Means of the results for all indexers with the 
three tools as % 

For group 1 (Second year students) the standard thesaurus and the list of descriptors return the same levels (29.6% 
and 29.5%), followed by the augmented thesaurus (25.9%). 



descriptors thesaurus thesaurus 


Figure 1: Group 1 - Second year students 

In group 2 (Fourth year students) the list of descriptors returned the best consistency results, 39.8%, versus 34.1% 
for the augmented thesaurus and 23.7% for the standard thesaurus, as shown in Figure 2 below: 













































List of Augmented Standard 

descriptors thesaurus thesaurus 


Figure 2: Group 2 - Fourth year students 

For group 3 (Fifth year students) the augmented thesaurus returns the best results (35.8%), followed closely by the 
list of descriptors (33.3%), as shown in Figure 3. 



descriptors thesaurus thesaurus 


Figure 3: Group 3 - Fifth year students 

From the results it can be stated that the list of descriptors provides the highest indexing consistency among all 
novice indexers, with 34.2% coincidence, versus 31.9% for the augmented thesaurus and 29. 2 % for the standard 
thesaurus, as Figure 4 shows. 



List of Augmented Standard 
descriptors thesaurus thesaurus 


Figure 4: All novice indexers 

Expert indexers (Group 4) also obtain their maximum consistency index with the list of descriptors, 55.7%. In second 
place are the results obtained with the standard thesaurus, 31.3%. The lowest consistency, 23.7%, was returned with 
the augmented thesaurus. This may be due to a lack of previous knowledge of this tool or to the fact that the scope 
notes annotate the meaning of the terms, leading the indexer to choose certain descriptors on the basis of the 
definition given and not according to previously conceived ideas. 
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descriptors thesaurus thesaurus 


Figure 5: Expert indexers 

We have also found the mean for all the consistency obtained for both expert and novice indexers. It is clearly 
observed that the tool with the best results is the list of descriptors (39.5% consistency), followed by the augmented 
thesaurus (29.8%) and, with a similar value, the standard thesaurus (27.7%), as can be seen in Figure 6. 
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Figure 6: All indexers 


With the exception of two cases, the highest consistency occurs for abstract Number Two. 

The results obtained in this study fall within the margins of consistency obtained in other previous studies, ranging 
from approximately 10% to 60% (Lancaster iq68 : Leonard 107^; : Funk and Reid 1983.; Markey 1984; Middleton 1984; 
Tonta 1991; Sievert and Andrews 1991; Iivonen and Kivimaki 1998; Leininger 2000 : Gil-Leiva 2001 and 2002 : Saarti 
2002 : Neshat and Horri 2006 : Gil-Leiva et al. 2008 and Kipp 2009). We also said in the introduction to this article 
that we can compare our results with the data achieved by Hudon (1998a and 1998b). Hudon also used three 
versions of a thesaurus - a) standard thesaurus, h) standard thesaurus with definitions for all their descriptors and c) 
stripped thesaurus with definitions, but hierarchical and associative relationships between terms, to see if the 
definitions including all descriptors of a thesaurus can raise levels of consistency among novice indexers. 

Hudon's results show that a) for the selection of all descriptors (main and minor), indexers who worked with the 
augmented thesaurus did not obtain better consistency than those who worked with the standard thesaurus; b) 
indexers that used the stripped thesaurus were equally or more consistent with each other than those who used the 
standard thesaurus. In contrast, for the selection of main descriptor indexers who worked with the augmented 
thesaurus consistency scores were better than the standard thesaurus in seven of the twelve documents, and the 
stripped thesaurus indexers were more consistent than the standard thesaurus indexers in eight of the twelve 
documents. Hudon concludes that the availability of definitions in a thesaurus does not increase the consistency in 
indexing of novice indexers in the selection of main and minor descriptors, and the availability of definitions in a 
thesaurus may lead to novice indexers achieving acceptable levels of consistency in the selection of the main 
descriptor when using a stripped thesaurus. 

In our research it has become clear that the list of descriptors achieved the highest levels of consistency in indexing 
for both novice and expert indexers. However, in the results obtained by Hudon with a stripped thesaurus with 
respect to others (standard thesaurus and standard thesaurus with definitions) there are some indications to suggest 
that the lists of descriptors can achieve similar results to standard thesauri and augmented thesauri. 

In any case, further research is required to corroborate these results. One important limitation of this research that 
should be pointed out is that in the two comparative studies we do not know what percentage of inconsistency is due 
to the complexity and subjectivity of indexing (reading and analysis of the document and selection of the appropriate 
keywords) or to the later selection of the descriptors of the controlled vocabulary (conversion of the selected 
keywords to descriptors of the controlled vocabulary); or how much inconsistency is due to the indexing languages 
used. 

Conclusions 

Inconsistency is an inherent feature of indexing, as we have seen from the data obtained in research conducted since 
the 1960s to the present day. Precisely because of the disparity of variables used in the research, it may be 
appropriate to carry out a systematic review and or meta-analysis of the relevant literature on indexing consistency 
to provide more light on this issue. Similarly, as has already been suggested, more research is needed on the 
properties of the lists of descriptors compared to standard thesauri or augmented thesauri with application notes or 
notes of definition, because in a small information system is always easier to build a list of descriptors than a 
thesaurus. Further examination of the properties of the lists of descriptors compared to thesauri could be a line of 
study to follow but with the inclusion of techniques using verbal protocols or ‘thinking aloud’ during the process of 
indexing the documents with thesauri. This technique will allow valuable information to be gathered on the use that 
indexers make of associative and hierarchical relations. 
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Appendix B: Abstracts 

Abstract 1 

ARAUJO RUiZ, J.A., ARENCIBLA JORGE, R. y GUTIERREZ CALZADO, C. Ensayos clmicos cubanos publicados en 
revistas de impacto internacional: estudio bibliometrico del periodo 1991-2001. Revista Espanola de Documentation 
Cientifica, 2002, vol. 25, n° 3, p. 254-266. 

The aim of this work is to assess the scope of the clinical research performed by Cuban scientific institutions. A 
retrospective search about clinical trials published by journals indexed in MEDLINE and Science Citation Index was 
carried out, and 172 references to works published with the participation of Cuban research centers were retrieved. A 
group of 653 Cuban and 175 foreign authors were identified. The average of authors by article was 7.16, and the most 
common author groups were made up of more than six specialists. A total of 82 clinical trials were the result of 
collaborations between scientific institutions; 83 research centers took part in the trials, 36 of them from others 
countries. The reports about the 172 clinical trials were published in 96 journals from 17 countries, and the 74,4 % of 
the articles were written in English. Sixty-three therapeutic products, techniques and procedures were tested in 
different types of patients,and 41 disorders were treated. Human adults, with a relative balance between women and 






men, were the subjects most frequently studied. The bibliometric study made it possible to confirm the Cuban 
advances as regards to the clinical trials execution for the authentication of products reached by the medical- 
pharmaceutical industry, as well as to define the research centers in the vanguard regarding this subject. 


Abstract 2 


CARO CASTRO, C., CEDEIRA SERANTES, L. y TRAVIESO RODRIGUEZ, C. La investigation sobre recuperation de 
information desde la perspectiva centrada en el usuario: metodos y variables. Revista Espanola de Documentation 
Cientifica, 2003, vol. 26, n° 1, p. 40-50.> 


User has been included in the research in information retrieval, and this factor has implied a new research approach. 
This perspective is focused on new issues related to searching process, such as formulation of queries, interaction 
between user and system, evaluation of the results obtained and influence of some personal characteristics. This 
exploratory paper examines 25 original research works of the user-centered perspective. A classification of the 
variables according to the following categories has been established: user characteristics, searching environment and 
process, and results. The coincidence in data collecting techniques, analysis methods and variables has served for 
checking the similarity between the research works analysed. Finally, a graphical representation of the different 
trends observed in these works is presented. 


Abstract 3 

ALCAIN, M a D., et a. Evaluation de las bases de datos ISOC a traves de un estudio de usuarios. Homenaje a Jose 
Maria Sanchez Nistal. Revista Espanola de Documentation Cientifica, 2001, vol. 24, n° 3, p. 275-288. 

The objective of this work is to approach the state of the art of the user’s studies about Quality Management and 
Evaluation of data bases, and to apply the existing models to a real case: the ISOC data base. To this end, two 
questionnaires have been designed: one addressed to end users and the other to reference librarians. The results 
show the differences between the two groups in the use, reasons for consultation, objectives and satisfaction. A 
difference in objectives and level of satisfaction has also been found between the two main users: researchers and 
professors on the one hand, and students, on the other. The results allow us to establish a map of the use of ISOC 
data base and to obtain some value indicators. It is concluded that ISOC is widely used and generally well valued. 
These kinds of studies have proved to be necessary in order to follow up users’ requirements. 
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Group 1: Second year students 

Controlled vocabulary used: List of descriptors. Consistency in % 
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Controlled vocabulary used: Augmented thesaurus. Consistency in % 
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Controlled vocabulary used: Standard thesaurus. Consistency in % 
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Controlled vocabulary used: Uist of descriptors. Consistency in % 
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Controlled vocabulary used: Standard thesaurus. Consistency in % 
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Group 3: FIFTH YEAR STUDENTS 
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Controlled vocabulary used: Augmented thesaurus. Consistency in % 
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